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Abstract 


A melhod for the detection and eBtlmotion of motion In Image aequencea la pre- 
aentedJn order to eatlmate motion parometera reliably, a gradient baaed apatlo- 
temporal conatraln equation for motion eatlmatlon ualng zero croaanga of wavelet 
tranaform la deacribed. Multlreaolutlon Image decompoaltlon la performed with 
the blorthogonal wavelet tranaform and motion parametera are hlerarchlally eatl- 
mated.Motlon vectora are alao eatlmated ualng Laplaclan of Qauaalan amoothlng 
filter, uang the aame conatraln equatlon.Flnally a performance comparaalon of the 
two methoda are carried out ualng the i^ynthetic and laboratory image aequencea 
reapectively. 
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Chapter 1 

\ 


INTRODUCTION 


Velocity information Ur important not only for detecting velocltleB and trajectories 
of objects but oiso os a ciue for image segmentation. It may be used to create 
new frames from the existing one’s through temporal interpolation. Therefore a 
new frame is usually created from two adjacent frames one in the past and one in 
future relative to the frame being created. A straight-forward approach for obtaining 
velocity information is to find the prominent features and track them by a matching 
method from frame to frame [1], The disadvantage with this method is that the 
computations required are very large and also if the initial velocity vector is wrongly 
estimated, it leads to on erroneous final velocity estimation. 

Another approach [2] uses the fact that for a moving objects the spatial and 
temporal intensity changes ore not independent of each other and ore ^ven by the 
relation: 

,, , ydl{x,y,t) dlix,y,t) . . 

dx dy m~ - ° 

It is assumed that the Image IntensltleB arc dllFerehliablc; Vg, and are the two 
components of the velocity vector; (dl/dx and dl/dy) are the two components of the 
spatial and temporal Intensity gradients; (dJ/dt) represents the grey level difference 



bfltwwn conuecullve framoB at a point (x,y) [2], 

Variation of the basic method [2] Involves processing of the Image before subtrac- 
tion. Fbr Instance In the boric detection scheme [3, 4] each image intensity ' I ' is 
first convolved with the Laplaclan of the Gaussian smooth! ng function (a?, y) and 

the aero crossings In the single frame of v®G(af, j/) ♦ / are located.The change In the 
value of ♦ / from one frame to the next Is non aero only when the edges 

have moved. This method con detect motion vectors correctly to some extent If the 
pixel displacement per frame Is small|of the order of one or two pixel per frame. The 
method however foils for large values of pixel displacement per frame. 

In this theris we propose a variation of subtraction method hosed on spatio- 
temporal constrain equation [3, 4]. This hierorchlol method for estimating motion 
parameters uses siero crosrings of wavelet transform [5]. A multiresolutlon image 
decomposition is performed using blorthogonol wavelet transform and the motion is 
ifierorchialiy estimated. It is shown that the motion parameters estimated ore reason- 
ably accurate and the method works satisfactorily for values of pixel displacement 
greater than 10 pixels per frame. It Is also shown that the spatial and temporal 
derivatives of the function (a<^i ^ he used to compute the component of 

velocity normal to the aero crossing contours. The computation of the normal com- 
ponent of the velocity vector is quite similar to other methods including the work of 
Morr and Ullmon [6] . 

1.1 Scene Segmentation 

In on image sequence, if all the pixel in a frame ore moving with the some set of motion 
parameters, then the correlation between two consecutive images is moximbed when 



Iho flrrt Imago la transformod according to the motion parametera. Thla la the reault 
of the well known fact that, given two Imagea, their correlation la maximlaed when 
one of them la a replica of other even If acaled hy a conatant. The altuatlon becomea 
complicated If there arc more than one moving object. In the correlation method It 
Ip obperved that multiple moving objectp give ripe to multiple peake correppond to 
the velocity of moving objectp. Thlp obpervatlon can be uped for detecting multiple 
moving moving objectp. 

Let UP puppope that there are two moving objectp In a frame, and the magnitude 
of Individual motion Ip pmall Thlp will contribute to two peakp. lype peakp are 
poppible if the magnitude of individual motion ip large, repultlng in aliaplng, thip can 
be avoided by low peep filtering of the Image. Another poppible pource of falpe peakp 
Ip due to merg^g of peakp depending on the relative broadnepp of the peakpJf two 
peakp of roughly the pame height are brougld together pufflciently elope, the pair of 
broad peakp bear a higher ripk of merging together to form a pingle peak.hbr the 
pame peparatlon the ripk ip leeo for a pair of pharper peakp to merge together to 
form a pingle peak. The High-papp filtering for example, can be uped to pharpen the 
correlation function. 

Now we have two contradlctoiy requirements On one hand, low-papp filtering 
ip needed to avoid falpe peakp and on other hand high-papp filtering Ip needed to 
avoid merging of peakp. l^laclng thlp dilemma, a pope&ble pulutlon ip a multlrepolution 
pcheme. Thip problem can be well repolved uplng Wavelet IVanpform', which decom- 
popep image into multlrepolution level. Thup a puitable pcene pegmentation can bo 
achieved uplng multlrepolution decompopltlon of Image, uplng the wavelet tranpform. 



1*2 Problem Statement 


In Ihlu a now method for detection and ertimatlon of motion with wavelet 
transform Is proposed. The Image sequences are decomposed Into various resolution 
levels uring blorthogonal wavelet transform and the motion vectors are hlerarchlally 
estimated using spatio-temporal constrain equation. The multiresolution decompo- 
sition of image is a pseudo frame; and the motion vectors estimated at the coarser 
layer are used to Initialise the estimation of motion vectors at the next resolution 
level. The motion parameters are hierarchllly estimated. 

A comparision of motion vectors evaluated using Laplacian of Gaussian smoothing 
function, for the same gradient constraJn equation was made with the hierachial 
method. The comparision showed that the hlerarchlal method of estimating motion 
parameters has overcome the drawbacks of Laplacian of Gaussian method. 

1*3 Organisation of Thesis 

The thesis has been presented in the five chapters. The first chapter contains the 
introduction and the summary of the work. The second chapter deals with the general 
theory of estimating motion parameters. In the third chapter wavelet transform Is 
briefly dlscusered. Chapter four discusses In detail the method of motion estimation 
using wavelet transform. The motion parameters were estimated using Laplacian 
of Gaussian smoothing function. The motion parameters evaluated using both the 
methods for small as well as for the large ^ue of pixel displacement per frame were 
analysed and compared In each case. Chapter five concludes the thesis and discusses 
the scope for future work. 



Chapter 2 


MOTION ESTIMATION 

2.1 Introduction 

The procepu of determining ihe movement of objects within the sequence of Image 
frames Is known as motion estimation. Procesmng Images accounting for the presence 
of motion, Is called motion compensated Image proceEslng 

Motion compensated Image processing has variety of applications. One appUcar 
tlon Is Image Interpolation. By estimating motion parameters we can create a now 
frame between two adjacent existing frames. 

In motion estimation problem we consider here, only translational motion of ob- 
jects. 


2.2 Various Methods 

Let and I{x,y,tf)) denote the Image intensities at and io respectively. 

We refer to and w PW(t and current frame. We assume 

i) ( 2 . 1 ) 

where dx and dy are horizontal and vertical displacement between <-i and <o. 



/(», 1 /, t) « /((a - V^{i - t.t),y - Vy{t - <_i)) <-!<<< <0 (2.2) 

A direct cooBequetice of equation 2.2 Is a differential equation which relates V* 
and Vj, to which is mlid in the spatio-temporal region, over 

which uniform motion is assumed. Tb derive the reiationship. Let 

«(»,?/) =/(ai,^,<-i) (2.3) 


From equations 2.2 and 2.3 


7(®,2/,<) = s(o(®,3/,0,K®>3/iO) (2.4) 


whore 

a(a;,ViO « a; - V,(i - t.i) and b(x^y^t) = j/ - V^{t - t^i) 
FVom above wo obtain 


dte 


da dm 


dy dady 
dJ{m^y,t) 

m 


^ as 

dbdx da 

(2.5) 

dsdb ds 

dbdy " db 

(2.6) 

ds da ds db 

da dt db dt 

(2-7) 


FVom 2.7 


^^dJ{m\y^£) _ 0/(aJj|/,i) 0/(a;,j/,i) 

eT" + ay + ~W~ 


= 0 


( 2 . 8 ) 


Equation 2.8 Is called the spatio-temporal constrain equation and can be gener- 


alised to Incorporate other types of motions such as zooming etc. equations 2.3 and 
2.8 are highly restrictive. For example they do not allow for object rotation, camera 
zoom, regions uncovered by translational motion and inultlple objects moving with 



different volocltloB, Hero It Is assumed that the background region Is not affected 
by object motion and the object Is moving with uniform transdatory motion between 
two Buccesslve frames under coninderation. 

Motion estimation problem can be classified broadly into two groups. 

1. Region matching methods. 

2. spatio-temporal constraint methods. 

Re^on matching methods are based on equation 2.1 ; and constraint methods are 
based on equation 2.8. 

2*3 Region Matching Methods 

Region matching methods Involve considering a small region In a frame and search- 
ing for the displacement which produces tlm best match among possible regions In 
on adjacent frame. In region matching method the displacement vector [dx^dy] Is 
estimated by minimising: 

Error = j j k) - /(«• y^ dy (2.9) 

or Errors f j 1 /(aciy.^D) - I (2.10) 

Jy 

The functions /(as, y, ip) ~ ^-i) cfi-hed displaced frame, difference.Tho error 

expression 2.9 and 2.10 Is zero at correct displacement [ci'c,tiy]. Minimising equation 
2.9 and equation 2.10 is a nonlinear problem .Attempts to solve this nonlinear prob- 
lem has produced many variations, which can be grouped into block matching and 
recursive methods, We discuss briefly block matcliing. 



2*4 Block Matching method 


One Btrdght-forward approach lo eolve the above mlnlmbation problem leto evaluate 
the error for every poBBlble [dx,cty] within gome reasonable range and choose [dx^dy] 
at which the error is minimum. In this method, a block of pUcl inton&itieB at to 
is matched directly to a block at time This is the beaiB of the block matching 
method. This method Ib computationally expcnmvc and many methodB have been 
developed to reduce computations. In the case of three step search method, in the 
first step, the error expresBion is evaluated at nine vaiues of (dx,dy). Among these 
nine values we choose (dx,dy) with the smallest error. In second step, we evaluate 
error expression at eight additional values of (dx,dy). We now choose from the nine 
values, that Is one previous and eight current values of (dx,dy), the minimum of 
(dx,dy). This procedure Is repeated once more. At the end of third step, we have an 
estimate of (dx,dy). This procedure can be easily extended to more-than three steps 
to estimate (dx,dy). There are other methods to solve the region matching problem 
[ 1 . 2 ]. 


2.5 Spatio-temporal Constraia Method 

Algorithms of this class axe baaed on the equation 2.8, which con be viewed os a 
linear equation of two unknown porometera (Vaj,V|/) under the aaaumptlon that 
Image Intenaltlea ore differentiable. 

In the simple aubtraetlon methods, the Intensities at each pixel at adjacent points 
In time are subtracted from each other; nonacro values In the resulting dlff'crence 
Image frame Indicate that something in the the Imago has cl^onged. It is assumed 
that these changes ore due to motlon,rather than the Illumination effects. Variation 



of 1hl» Involve procesBlng oflrtingo before subtraction [3]. 


2.5.1 Lapladan of Gaussian Method 

In 11110, the image intenfflty field is first convolved with Laplacian of Gaussian 
smoothing function V^G(atr, y) and the zero crossing In the angle frame V*G(a[r, 3 /) #/ 
are located. The change In the value of y*G(a:,y) * 1 from one frame to the next at 
the location of zero crossings in the current frame are nonzero only if the edges have 
moved. If no motion has taken place and there is a change only in illumination, then 
the change in value of V*G(af,y) * / from one frame to the next at the location of 
zero crossings in the current frame will be zero. I'hus this method can discriminate 
between motion and tune varying illumination. FJquation 2.8 can be u«*d to compute 
the normal component of velocity vector [3). I’bo magnitude of the nonnal component 
of velodty 'tin’ at the zero crossing Is given by [3]! 
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~My*W 


( 2 . 11 ) 


and the angle '& between the unit normal to the contour and the poBltlve x-aode Ib 
given by 


6 — iafi (2.12) 

ai 


2.6 Image Interpolation 

One of the appUrntlon of motion estimation Is In Irnngo Interpolation. Imago Interpo- 
lation can be used in changing the size of a digital Imago to Improve its appearance 
when viewed on a display device A eequenpo of Imago frames can also bo Intorfxilatod 
along the temporal dlmension.A 24 frames/sec motion picture can be converted to a 



60 flold/BCC NTSC signal for TV through Interpolation 1>.rnporaI Interpolation can 
also bo UBod to Imprcfvo the appearance of slow motion video. 

Interpolation can also be used in other eppItcotloruB such oh image coding.Por 
example, a simple approach to bit rate reduction would be discard some pixels or 
some frames and recreate them from the coded pixels or frames. 

2.6.1 Spatial interpolation 

Consider a 2-1) sequence /(«j,n 2 ) obtained by sampling analog jdgnal /^(a:,?/). 

/(»i. fh) = y) I,,. (2.13) 

CO DO 

12 f(ni,n2)h(x-niQi,y- ftzTi) (2.J4) 

ni«--co t)8*-~oo 

where h(x,y) is the Impulse response of an ideal separable analog low- pass filter. 

There are several dlfhcultlos In using equation 2.14 for Image interpolation. I'he 
analog image /n(iv,y) even with an antialiasing filter, is not truely band-lirnited, so 
aliasing occurs when /e(x,y) is sampled. In addition h(x,y) is an infinite extent func- 
tion, BO evaluation of fd^^y) using equation 2.14 cannot be carried out in practice. 
IV) approximate the interpolation, one approach is to use a low pass filter h(x,y) that 
is spatially limited. For a spatially limited h(xiy) the suimnation in equation 2.14 has 
a finite numbers of nonziero terms. If h(x,y)iB a rectangular window function given 

by 

-Cn/2)<x<{TM - (7^/2) < 2/ < (n/2) (2.15) 

then it is called a zero-order Intorpolation.ln zero-order Interpolation, /c(a’, 3 /) Is cho- 
sen as at the pixel closest to (x,y). Other examples of h(x,y) which are 

more commonly used are functions of smoother shape such os the spatially limited 


Gaussian function. 



Another simple method widely used in pradlco Ib blllnee.r Interpolation, In this 
method, I b evaluated by the linear comlunatlon of /(ni,nj) at the four cloBest 
pixels. The Interpolated the bilinear Interpolation method is: 

}c{^>v) = (1 - A„)(] - Ay)/(«i,«i) (2.16) 

+ (] - Ag.) Ay /(nj[,ni + 1) 

+ Ag, (1 - Ay)/(ni + l,n2;) 

+ A*Ay/(»i + ],«H-l) 

where A#, -(at- nyP\)/T\ (2.17) 

and Ay » (y - n-^'-^/Ti (2.18) 

Bpatial interpolation Bcheme can also be developed using motion CBtimation al- 
gorithms, One example, where an image frame that corisistB of two image fleldB is 
constructed from from a single Imago field Is discussed in chflpter(4). 



Chapter 3 


WAVELET TRANSFORM 

3.1 Introduction 

An imporiani prablcm in fagna>l procesping ip io define a repreperrtaiiion lhal is well 
adapted for extracting the Information content of pignaJp. The pharp variationp of 
signal amplitude are generally among the mopt meaningful featurep.FV>r example the 
dlpcontlnultlep of the Image Intenplty provide the contours of the different objects. 
When the signal Includep Important ptructurep that belong to different pcalos it ip 
often helpful to reorganise tbe signaJ information irrto a set of detail? componontp of 
varying tdsse. The wa^felet transform is the linear operation that decorti{)opep a signa.! 
Into components that appear at the different scales. Thip transform Is based on the 
convolution of the signaJ with a dilated filter. 

3.2 Waveletsi A Brief Review 

The wavelet transform of a signal f{t}v& by definition its convolution with a wavelet 
w(t) dilated by a factor (&)\ 


V^'/(o, i) 


IT 


( 3 . 1 ) 



^/(o,fc) *= (3.2) 

which iBoquivalont 1o iiHcrlng the tngnal /(<) with the bn tidpapa filter W^(my), whoso 
bandwidth changes according to the scale pn-rameter o. Clearly , largo sceles cor- 
respond to narrow srnoothing filters that represent a global view of the signal f{i) 
and small scales correspond to wide flltprs that look into the details of f(t) (i.e. high 
frequency components). 

Wavelet expansion, of the signal f{i) is essentially a decomposition of its frequency 
content using filters of constant relative bandwidth. The signal f{t) can be recovered 
from its redundant wavelet transfonn coefficients using: 

assuming that 


lw{i)di ^ 0 (3.4) 

As it Is the case with the Riurler transform, where a signal is expanded in terms of of 
complex exponentials of different frequencies, a wavelet expansion involves dilations 
of a single wavelet (“mother wavelet”). I’he choice of a mother wavelet depends on 
the application, where a particular wavelet is chosen batjf'd on its time and frequency 
localisations. 

Orthogonality is an Important element of wavelet analysis, and a mother wavelet 
Is orthogonal to Its own dllH,tioriB and translations. Wavelets provide orthonormal 
basis for expansions of functions that are not of single frequency and are therefore 
ideal for characterising signals with discontinuities. 



I'he wavelet trauBfonin parpnieters can bo dlBcrotiBcd P»t that 



0 r. aj* 
b = nap' 


(3.6) 

(3.7) 

(3.8) 


and T 1b the Bampllng period. The algnal f(i) can bo rocoverod From Itp expatiBlori 
coefHciontp lining 


= (3,9) 

■where A la conatajit. 

The cape where oo = 2 Ip known as the dyadic -wavelet transform, where the slgnsJ 
f(t) 1b bafid-papB filtered using octave band flltere This type of -wavelet bos the form 

= 2 {2-^k - n) (3.10) 

m^n a Z 

The discrete wavelet transform (DWJ') of discrete time Bequence f{k) is esBcn 
tially a multlrcBolution characterlBotion of f{k) . Generally we take the DWT of a 
signal that is both time limited and rebolution limited. A continuous time signal , 
uniformly sampled satisfies this criterion, [8]. A dyadic discrete wavelet transform is 
essentially a decomposition of the spectrum of f{k) ; F(y}) into orthogonal subbands 
defined by 

¥t ¥fnW' 

3 ■= li2, ,t/ 


(3.11) 



whoro T Is tho sampling period associated with f(k). 


3.3 Zero CrossingR of Wavelet IVansforiu 

If the wavelet function is a second derivative of a smoothing function '0,(a:)' 

, then the zero crossings of the wavelet transform correspond to the local 

variation points of the signal In an Image signal let J(x,y) be the representation 
of the Image and be the smoothing function, which is used to smooth the 

Image. In the case of separable smoothing function 

e(;x,y) = ${x) 9{y) 

Two wavelet functions are given by; 

t^.,v (® , y) “ iy)0. (sr) 

The two wavelet transform are given for horizontal and vertical direction os 

(3.16) 

If the smoothing function is Gaussian, than the detection of of zero crossing points 
of wavolct transform is equivalent to extraction of edges with Laplocian of Gaussian. 

The DWT is implemented using a bank of biuidpaf.s and low pass discrete time 
filters, h and ff , As the input sequence /(ft) propagates through the filter bank tree 
of low pass and high {.muss filters , the output of the high pass filter g at stage m 
represents the detail signal which Is a sampled version of the wavelet transform of 
/(ft) at tho scale 2"*. At each stage the bandwidth of iiotb filters Is halved with the 
upper half band associated with tho high pass filter g and lower half band ossociated 


(3.n) 

(3.13) 



wllh thp low paETB firter h. 1’h« resuHIng eigrifil drcfiiapoHltiori 1 b 

■tho DWT of /(/t) l.e. Co- Roeoluilon level ni 1b inclutlfirl for the seke of ctJoiploionoBB, 
'J'he fiUerB •thus each perforia a convolullon end fleclrrioUori on Input The Input 
Blgnal Cb = fQc) Is called reBolution level zero. Ilewdulion levels m - 1 and in arc 
c"*"^ and cf” respectively, and tf" is the din'ercnce level m It is celled difference level 
because It represents the difference In the signal between c'"' ^ and o™. Since DWT so 
Implemented Is a linear system, a norma) process at Input results ins normaJ process 
at output. 

Finally due to discretisation of the wavelet trarisform parameter 6, the wavelet 
expansion coefficients are no longer shift Invariant. I’he expansion coefficients of 
a signal /(d) may differ from those of /(i ~ <0) . Indeed the wavelet coefficients 
of a particular pattern arc modiHed when the position of this pattern is thangcd.On 
contrary, It Is clear that the position of the Ksero- crossings of a <^dlc wavelet transform 
are translated when the signal /(s) Is translated .This property of the jieru crossings, 
If the motion hft.s taken place, is used for detecting motion of the uhjei Ih 



Chapter 4 


MOTION ESTIMATION WITH ZJ5RO 
CROSSINGS OP WAVELET TRANSFORM 


4.1 Introduction 

3n Of del* io aoourfliiyiy efrtlm6;tfi motion many mtithodn have been proposed and 
tOBted.Theae methods tan be cliteBifled as Matching smi Spatio-tompofal gradient 
based methods. Although conventions! block matching methods can estimate motion 
using relatively simple calculations, they are inadequate in some cases. I'’or example, 
if block boundaries and the object boundaries do not coincide. Jn such cases the 
blocJc siae of the two cosecutlve frame should be so chosen such that It eonta.ins the 
object. The Block matching method considers a small region In a frame and searclies 
for the displacement whicli prodtR»;s the best match among the possible regions In 
an adjacent frame. In tlds method the displaccniont vector [iia5,fi^] is estimated by 
using the equations (2.9) and (2.10) given in the chapter (2). 

The gradient based method works quite well If the magnitude of the motion is 
not large. To estimate motion parameters, which takes care of this problem in grsr 
dlent based method, a new hlerarcbial motion estimation approach Is proposed. It is 



shown 1110,1; when molion parameters ore estimated hicrorchiolly utnng gradient bosod 
method at each resolution lovcl, it gives a bettor estiiiiotion of motion pararnotors 
than the l> 0 placian of Qauwian Method which is siiitoltle only for small displ0.co- 
ment per fr0tne. A similar hierarchkl method for motion estimotion was used by ( 
Cheong and Alzawa) [ 7 ]. They UBcd the block trmtehing technique for motion OBtiinftr 
tion, However interdopondemrr of layers Is a serious drswhax'k for estimating motion. 
Here Interdependency of layer meens thet the motioti {^ireyneters calciilsted fl,t the 
coarser layer Is tiscrl to minimize the error expression at the next detail layer for 
estimating the motion parameters. In this way the motion vectors ore hlerarcliiolly 
estimated. However If the motion estimation at the coorsur layer is incorrect It is 
almost hflpossible to overcome the misleading esllmatlun in the next delail layers. 

If the wavelet function ‘^,{1:)' is chosen ss the second derivative of a smoothing 
function ^5,(11:)' , then the aero crosrfngs of thi* wavi'li l iiayiHfoftii W,(v’) con osi->ond 
to the local variation points of the signal f(x). In ati image tdgnal let l(x,y) bo the 
representation of the image and be the smoothing function, which is used to 

smooth the Imsge. For separable smootliing function 

e(j£^y) = e(pc) 0 {y) 

Two wavelet functions ore obtained os: 

= ( 4 . 2 ) 

Tho wa-volot transforms for horiiionta.i and vortical direction are glvon by: 




( 4 . 3 ) 

( 4 . 4 ) 



The firsi {rtcp of motion estimation is, scgincntation of the ima^e into stationary 
and moving areas Onro the stationary pixels are krumn they can be ignored in 
the following stage of motion esliniation. Rcgiucutation is obtained by applying the 
simple tlircslioldirig procedure 

Moving If 

I h{x,y) - h{x,y) I >'J\ (4,5) 

Stationary If 

\ - h{3’.,y) \ <7*1 (4.6) 

Whore '7^ is the threshold chosen by the user, depending upon noise, image 
Intensities etc. 

On evaluating the isero crossings of wavelet transform ; the difference of zcfo 
crossing will be non zero If and only if the image has moved. Although the wavelet 
coefficients of a particular pattern are modified when the position of this pattern are 
changed^ yet the position of the zero crossing of a wavelet transform sre trajislated 
when the signal f(x) is translated. The extracted zero-crossing points represents the 
features of the image, but these points seldom charactorlso the objects boundaries 
completely. But here we are only interested in the zero crossings. 

4.2 Hierarchial Motion Estimation 

I*br a given imago sequence, raultiresolutlon image dccomiwsition is obtained by using 
a biorthogonal wavelet transform. The dydir -wavclel used for the wavelet 

transform and the corresponding impulse response coedTirients of filters 'If and ’G’ 
corresponding to the wavclot is shown in fig (41) and Mbldc (4 1). 



The rnagnltude of the mollon vcrtor coMipuied ai co!i,if,<'r layer in tlie rriultireBo- 
lutlon repreaentation H,re UKed to initinliBe the (iRtirnfi.tlti)i of magnitude at the finer 
layer. Lot bo the magnitude of the avomgo mlue of the motion vector computed 
at rcBr>Iutiori 2“'^.ln the next resolution the Dj is taken to initialise the ca.L 

culation of motion vector magnitude At the next resolution that is 2' Dj 
and i); 4 i both the values are used and the value of luaj’iiitudo of {notion vector cal- 
culated at each pixel are compared with both 1)j and /J, u* ^’InaJly at the finest layer 
the magnitude vector is calculated by comparing with the motion vector magtiitudes 
of the two previous layers. The angle is calculated at the finest layer. 

The steps involved in the motion estimation can be sutnmerlsed as under: 

1. First the mvclet tranKforrn of the imogc soquenco frame Is taken. For each 
Image frame, the Intensity Is convolved with second derivative of smoothing 
function or the vvavclot function. 

2. 1'ho wavelet transform decomposes the imago into difierent resolution levels 
containing coarser to detail mformations as the resolution level is increased. 
The Imago of the same resolution level Is separntod out in each frames for 
evaluation of motion parameters hierarchinlly. 

3. The estimation of motion vector D, at coarser layer 2“^, obtained as follows; 

(a) The image scqucriccs at the coarser layer 2"^ in two successive frames Is 
taken. 

(b) The temporal difference of Image sequence at this resolution step is cab 
cuinted. 



(c) Gradiolii of Imtigc dl/dx tind dl/d^ It. rtiloulptf'd uBing a’3*'3' Sobol opcr- 
alor. 

(d) Calciilalioii of nmgnUudeof motion vector and direction of motion a.t zero 
croBEfing is calcukled by: 



(4.7) 


and the angle '0' bctwicn the unit norma,! to the contour and the poBltlvc 
x-iwiIb is given by 


ft/ 

0 — tttu 


dtp 


(4.8) 


(e) Finally the values of Un and angle 'O' Is averaged out for which are not 
aero. 


4, At resolution 2' ^‘•^^that la th« next higher resolution; the above procedure is 
repeated. But for magnitude calculation in (d) above, each value of magnitude 
1b compared with Dj and a-ll values Icbb than Vj aio discarded. Then average 
of all thcBe vaiues thus calculated is taken. Let thits bo Dj\i. 

6. At the next layer the magnitude of the motion is compared with Vj 

and Dju Blraultaneously.That iB,all /nagnitude values which arc greater than 
are kept, and if no values are greater than D, n than values grca.tcr than 
Dj are kept and finally average of all these values thus calculated is taken. 

6. In a Blmllar way the magnitude of motion vector at the finest layer is calculated. 

7. Similarly, Dirwtion vector is calenlatcd a.t the finest layer as calculated in 3(d) 
and average Ista-ken of all the valuemso calculated which arc not zero. 


I 1 T KANPUR 



The mo'tlon voctorewore also calculaicd U6.irig Laplaciau of Gaupsian etnoothing 
fundlon.Tho Btops bvolvod arc foUowB: 

1. The Image inbriBity 'P is first convolved with Pw-plarianof Gairssian Brnoothlng 
function. 


2. The temporal difference of Jrnftgo sequence is calculated. 

3. Gradient of image dl/dx and dl/dy Ib ca.)culR,ted using a ’3*3’ Sobcl operator. 

4. Calculation of raagnitiule and direction of motion at nero crowjirig it? carried out 

by 

i! 

and the angle botwoon tho unit normeJ to the contfuir and the poBitivo x-axip 
If given iy 


0 <o»r^ 


&L 


(4.10) 


BS6 


6. Finally the values of Un and angle '9^ ib averaged out for the values which are 
not aero. 


4.3 Experinriftiil Kesulis and dkeussion 
4.3.1 Synthetic Image Sequence 

A Moving Bquare.A synthetic imago sequence used in tliis eaperirnent conHistfld of 
a brigiil square on a. dark background.'J'he total image sbe as 256*256 pixels. I'hc 
square started In tho upper left corner and moved diagonally towa.rds tho lower right 
corner by J .4 U pixel /frame fig (2). Magnitude and dirticlion are calculated utting the 



l/ftpIaciB,n of GauBHlnn and elgo by murtl-resolutlon dwomposiUon of image soqurnre 
framoR UBlng wavelet tranufomi of Boqimnrn frwiioB. 'J’ho obKormilonB are* 


1. t;op!fwian of GouRBian. 

MagntUido 

AetufiJ Caloidn.-ted Error 

(a) Elrsl and Second Eramc. 

1.414 0.7 0.7 

(b) First and Third Frame. 

2.828 1.4 1.42 

(c) Second and Third Fratnn 

1.414 0.7 0.7 


! 

Direction in Degrees 
Actual Calculated Error 

135 159 24 

135 159 24 

1.1.5 1.59.10 24.16 



2. Wavelet IHrisform of Image Sequence. 


Magnitnrle Dliocllon in DogreoB 

Actual Calculated Error Artnel CalculftWl Error 

(a) First and Second fVaree. 

1.414 1.422 0.01 13.S 134 01 

(b) First and 'I'hird Finnic. 

2.828 2.862 0.04 13.5 134 01 

(c) Sccriud and I'liird Frame. 

1.414 1.457 0 04 135 138 03 



4.3.2 Laboratory Image Sequence 

Bocft-UHO of ^ho Ihnilatlon in equlpmeui it was uot possible to obtHin roa.! time im- 
ago sofiuences of natural scone. Instead a Blationn.r,y ca-rrietu. wa* ustd to view a 
model of moving circle. This circle oven contains fine srolc structures fig (4.2). iVo 
such frames were taken for eBtImation of motion p)arnnieters. I'hc followings arc the 
observations: 

1. Laplocian of Gaussian. 

Magnitude 

Actual Calculated Error 

(a) First and Second tVar/ie. 

12.69 3.3 9.39 

2. Wavelet iVansforin Image Sequence. 

Magnitude 

Actual Calculated Error 

(a) First and Second E-ame. 

12.69 12.622 0.U7 

The followings are the observations: 

1. Laplacian of Gaussian can estimate only siotdl values of motion vector, and 
failed to estimate larger values of motion vectors. 

2. Error in estirtiated values of motion parameters ; when Laplacian of Gaussian 
method was used wore quite large, both for magnitude and direction vectors. 
Whereas the hlorarchlaJ wavelet transform motluol worked quite well in esti- 
mating small as well as larger values of motion ver tors. 


Direrlion in Degreos 
Actual Calculated Error 


18.43 23.9.^> 05.52 


Direction in Degrees 
Actual Calculated Error 


18.43 17.90 00.63 



3. In normal gracliciU based method the values' of dl/dx and dT/dy is wro or 
very Krnall in hornogonoous regiun, H Ir difrieiiH to eullrnate the velocity tliero. 
I’horofor the plxeU whose grorlient value | dl/dx -1 dt/dy | is greater than a 
particular threshold is taken Into conslderatiftri frtr « aleulations. I’hat is manual 
throBhoIdlng is rccjulrod. But In blera-rclilal method of wavelet transform this 
1b taken care of autotnatlcaJly through ItiitlaJlBatiun an we move from coarser 
layer to the next layer, 

4. 1’ho number of frauics required for reasonable correct estimation of motion voc 
tors in the case of Laplacian of Gaiusian arc quite largo in numbers. Whereas 
in multi" resolution wavelet transform pseudo frames arc created as one Image 
frame is decomposed into various layers of image and reasonably accurate esti- 
mate can be carried out using just two frames’. This is shown cxperlrnontally. 

Here wo see that since the pixel displecoment per frame waa lorge the conventional 
gra^lient ijased method using Laplacian of Gaussian ns a smoothing function failed to 
estimate the motion vector parameters. Whereas multi resolution of xero crossings 
of wavelet transform could ostirnn.te the motion paraiuclers quite accurately. 

4.3.3 Application of Motion Estimation Method to Spatial Interpolation 

I'he spatial interpolation technique may be used in creating a frame from a field, but 
incorporating some additional knowledge about image may improve the performance 
of a spatial interpolation aigorithms.Let /(ar,v i) and /(a', 2 / 0 ) denote image intensi- 
ties of two adjacent horizontal scan liucs of afield.Wo wish to create a new horisiontal 
scan lino between /{x^y 1 ) and /(w.a/o). One model that takes into account the 



Bpatfft.! coritInuUy of eurh olcoiontB m coirtours encl Bonltbcs \b. 


/(®>1/d) “/(dp ^ttiV i) (^’^0 

Wboro dg IB ^hc horimointa! displnroinont botwetm y j end ^/o Equation 4.1 1 cejn bo 
vlowed ftB e Biwrla) ctine of uniform trariBlationn! velocity model equation 2.3 The 
Bpntlel variable ‘y’ In equation 4.11 has a functinn vciy eirnilar to the time varlahlr 
’t’ as In equation 2.1 and there is only one Bpatkl variable in 4.11, while there 
are two spatial vaiiablo V and "y’ in equation 2.3. Ap a result the problem it? now 
of CBtimating dg in equation 4 11. Eor examine under the asbumption of uniform 
velocity, equation 4.11 can be ctxpicBtJcd as: 


=■ /(» ‘ K(y - y y i<y<vo 


v>hioh leads to 


’■ fe ~ ■ 0() 


(4.14) 

C4.J3) 


Thus once the motion pararacters arc cptiniatcd these can be uRed for Ri.?i)tia.l inter- 
poktion. This is shown in fig (4.10) and fig (4.11). I'be new fra, me is created with 
the knowledge of motion paremKers 



liable 4.1. Wavelet Kilter Coefllcierxtb Uecd for CaltulatioiiK 

11 G 

0 0.4347 0 71 18 

1 0 2804 -0.2309 

2 0.0450 -0.1120 

3 -0.0393 -0.0226 

4 -0.0132 0.0002 


6 0.0032 0.0039 






Figure 4.?: Tefcil Itringo of OiuUi 




FigUN! 4.3: hniigtiH of Oiftlc "JW JiVatiiMK Sitlo liy Sido 



Figure 4 , 4 , Su[H'tItniHtH'’'i Himwiiig 1)ihli«'t'itU!til of Circle 






Figui'ii 4.5: Wavdcl 'J 


5it(.lo Jiria.{ 5 i: ft.l DilTurctii llwrolutlon 












Figure 4.8. Jtnagc of Ihe Superitnfwifjod 1'VH.tiu^b of SqtiHri' fihowltig Difjplflofirtient 




Flgui'ti 4.0: 'JVHtt 



Ihifti't) urSquai**! ftl DiirKi-tint lU’Holuiiun 





Fipiii'ft 4.10: llrconfitnioleil ftfOiidt! 







Flguri) 4.J1: llocotiBtnicWl I'Vtuiio of^rjiinro 



Chapter 5 


CONCLUSION 


In ihip work the hlorarchial mo-thod of moliori cptltiifitlon using aero croKBdngs of 
wavolot transform ip propop«jd. Tiu) gradient based constrain equation ip used for 

t 

motion optlmation.Tt Ip shown that thip method ellmhiatciH the drawbacks of block 
matching method B,p well ap that of the convetitionaJ grndienl tnethod, 

Block matching method falls if the inltip.l cptlrnntlon ip -wrong nnd ip compti 
tationally cxponplvo. I'he convontlona.1 gradlotit bflpixl method using Lfipkclan of 
GatiFrian pmoothitig futtetion -works well for small valuop of pixel displacement {>or 
frame. Whereas the hlerarchlai tnethod prepeiiled (un, not only estimate motion 
vectors Rccuratoly but is snUablc for large vahics of pixel dipplBcernont per frame. 

6.1 Scope of Future Work 

The scope of future -work In this area Is plenty First of all the method can be 
extended to cater for zooming and rotationary movements of objocts.lt can aipo be 
extended to giobaJ and local detection of tnolion with various objects moving with 


difTorotit velocities. 
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